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ABSTRACT 


This  paper  is  a  revision  of  WP-2888.  It  reviews  the  present  state- 
of-the-art  techniques  of  narrowband  (4  KHz)  wireline  equalization  for 
PAM  signals.  Two  basic  minimization  techniques  are  discussed.  Related 
derivations  and  proofs  are  included.  In  addition  to  linear  distortion,  the 
problems  due  to  phase  jitter  are  mentioned  as  areas  of  possible  future 
research. 
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Sampled  Data  Model  of  System 


SECTION  I 


1 . 0  INTRODUCTION 

In  the  relatively  noisefree  (S/N  >  26  db)  environment  of  wire- 
line  data  communications,  linear  distortion  in  the  transmission 
system  is  a  major  concern  when  trying  to  achieve  high  information 
throughput.  Data  pulses  sent  through  such  a  transmission  line  will 
Lend  to  overlap  upon  their  neighbors  causing  intersymbol  interference. 
To  minimize  this  distortion,  equalizers  consisting  of  tapped  delay 
lines  with  adjustable  tap  gains  have  been  considered  ^  . 

Another  serious  contribution  to  signal  distortion  is  the  presence 
of  incidental  phase  modulation  caused  by  the  transmission  system's 
carrier  multiplex  equipment.  The  purpose  of  this  report  is  to 
discuss  the  basic  concepts  and  techniques  of  signal  equalization 
for  multilevel  PAH  signals  in  order  to  develop  and  quantitatively 
evaluate  proposed  signal  processing  techniques  (primarily  digital) 
that  are  intended  to  cope  with  the  above-mentioned  signal  distortion 
e  f  fee  ts . 

The  discussion  given  here  is  concentrated  on  time  domain  digital 

equalization  techniques  which  make  use  of  tapped  delay  lines  with 

appropriately  weighted  tap  gains.  Presently,  equalizers  are  designed 

so  that  tap  gains  are  adjusted  using  adaptive  algorithms  that  operate 

vhile  data  is  being  transmitted.  These  methods  have  been  shown  to 

[8] 

work  better  than  preset  algorithms  and  are  relatively  easy  to 
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implement.  Thus,  the  problem  of  linear  distortion  can  be  handled 
without  much  difficulty. 

The  problem  of  compensating  for  phase  jitter  (modulation),  on 
the  other  hand,  has  not  yet  been  solved.  Phase  jitter  causes  one 
to  sample  at  non-optimum  times  when  distortion  is  enough  to  greatly 
increase  the  error  rate.  The  nature  of  phase  jitter  is  examined 
in  terms  of  time  and  frequency  domain  response  and  interaction  with 
the  modulated  data  signal.  Techniques  being  used  in  newly  developed 
modems  are  described  and  their  limitations  are  considered.  Thus, 
further  consideration  must  be  given  to  techniques  for  controlling 
sampling  timeL^J  and  ''tracking  out"  phase  jitter. 
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SECTION  II 


2.0  OPTIMAL  TRANSVERSAL  EQUALIZERS 

Figure  1  illustrates  a  transversal  equalizer.  Its  output,  y(t) , 
is  a  weighted  sum  of  delayed  (or  advanced)  values  of  input  x(t): 


n 

y(t)  =  £  c^x(t- jT)  ,  (l) 

j=-n 

where  x(t)  is  the  delayed  value  of  the  input  signal  that  appears  at 
the  center  tap,  T  is  the  amount  of  delay  between  each  tap,  and  lcj] 
is  the  set  of  N  “  2n  +  1  tap  gains.  Because  our  concern  is  with 
the  sampling  instant,  one  can  rewrite  equation  (1): 


n 


-  E 


C  .X  . 
J  k-j 


(2) 


j=-n 

where  y  =  y(kT)  and  x,  .  =  x(kT-jT).  Assume  that  x(t)  is  the 

K  K  “  J 

result  of  sending  a  single  pulse  of  unit  amplitude  through  the 
system.  Define  h(t)  as  the  output,  y(t)  ,  that  consequently  comes 
from  the  equalizer.  Ideally,  we  want  the  sampled  values  of  h(t)  to 
be  given  by: 


k=0 

k*0 


(3) 
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Figure  I  TRANSVERSAL  EQUALIZER 


However,  having  a  finite  number,  N,  of  taps  permits  a  suboptima  1 

solution,  satisfying  equation  (3)  for  at  most  N  values  of  k.  Lucky 

I  71 

has  shown"  J  that  if  the  eye  pattern  for  x(t)  is  open,  the  optimal 
solution  for  N  taps  is  given  by: 

n 

uk  =  T.  cjVj  »  iki  ^ n  ;  (4) 

j=-n 

where 


_  «  1  ,  k=0 

\  l  0  kft) 


(5) 


In  matric  notation: 


u  =  X  c 


(6) 


where  u  and 
are  samples 
eye  of  x(t) 


c  are  N-vectors  and  X  is  an  N  x  N  matrix  whose  components 
of  the  response  of  the  system  to  a  unit  pulse.  If  the 
is  initially  open,  then  equation  (6)  can  be  solved: 
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where  X  is  the  inverse  of  matrix  X.  If  the  eye  is  not  open,  X 
may  not  necessarily  exist  or,  if  it  does,  equation  (7)  may  not  yield 
the  optimal  solution.  Should  the  former  be  true,  one  might  try  to 
minimise  the  square  of  the  distance  between  X  c^  and  u: 


L 


4 


(X  c  -  u) 


(8) 


•where  superscript  "Tn  indicates  a  matrix  (or  vector)  transpose. 
L  is  minimised  by  setting 


(9) 


to  get 


T  - 1 
c  =  (X*X) 


T 

X  u 


(10) 


Note  that  equation  (10)  reduces  to  equation  (7)  when  X  is  non¬ 
singular  (i.e.,  X  ^  exists). 
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SECTION  III 


J.9  ITERATIVE  PRESET  ALGORITHM" ' J 

An  iterative  algorithm  is  necessary  in  order  to  average  out  any 

affects  of  noise  on  the  x(t)  measurements.  An  iterative  algorithm 

\  7 

similar  to  Lucky* s“  J  requires  the  transmission  of  a  series  of  test 
pulses  that  are  spaced  relatively  far  apart  so  as  not  to  overlap. 
Define  error  vector  £: 

q  ^  h  -  u  ,  (11a) 

or 

4  f  \  .  kpo 

qk  '  1  h‘C  -  J  ,  k=0  .  (11b) 

0 

Alter  each  Lest  pulse,  the  tap  gains  are  changed  as  follows: 

Ac.  =  -  e  sgn  q.  (12) 

J  j 

where  e  is  the  smallest  increment  one  can  make  for  a  tap  gain. 
Convergence  of  the  algorithm  given  by  equation  (12)  is  proven  in 
Appendix  A. 
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3. 1  Adaptive  Version  of  Preset  Algorithm 


[8] 

The  adaptive  version  of  the  preset  algorithm  is  one  that  makes 
an  estimate  of  each  q^  during  a  period  of  data  transmission  and 
then  updates  the  taps  at  the  end  of  that  averaging  period  according 
to  equation  (12).  Appendix  8  shows  that  the  estimates  are  made  as 
lo 1 lows : 


K 


=  —  r 

KS  L. 
k=l 


'  ks  a  Vk-i 


(13) 


..here  K  is  the  number  of  samples  used  to  reach  a  decision,  S  is  the 
average  signal  pov^er,  a  is  the  amplitude  of  the  kC"  message,  and 

1C 

e.  =  y,  -  a  where  y  is  the  k^  output  sample.  Note  that  if  the 
K  K.  K.  K. 

eye  is  initially  closed,  errors  will  be  made  when  deciding  upon 

the  a^'s.  Consequently,  the  "q  's  may  well  be  incorrect  and 

convergence  may  not  necessarily  occur. 

[8] 

3.  2  Digitized  Adaptive  Equalizer" 

Lucky  simplified  the  previous  algorithm  considerably  by 

digitizing  and  allowing  its  rate  of  tap  gain  update  to  be  variable. 

Rather  than  correlating  analog  data,  as  in  euqation  (13),  the  digital 

algorithm  employs,  for  the  ith  tap,  an  up-down  counter  that  counts 

up  ^hen  sgn  e,  =  sgn  a  .  and  down  when  sgn  e,  ^  sgn  a  ..  When  a 
k  k-i  k  K-i 

counter  reaches  one  of  its  two  thresholds,  it  will  cause  its  tap 
gain  to  either  increase  or  decrease  by  e,  depending  upon  whether 
the  lower  or  higher  threshold  is  crossed;  upon  reaching  a  threshold, 
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it  is  reset  to  the  center  of  its  range.  The  probability  of  correct 
update  (estimate  of  q^)  as  well  as  speed  of  convergence  is  determined 

by  the  thresholds  of  the  counters, 

[si 

Lucky  has  shown w  J  that  for  a  given  amount  of  accuracy,  e,  this 
digitized  equalizer  converges  faster  than  the  one  mentioned  in  the 
previous  section.  This  occurs  in  spite  of  the  use  of  the  signum 
function  to  discard  much  information.  Use  of  the  non-linear 
sequential  test  in  the  digital  version  causes  taps  to  update  much 
more  often  than  every  K  samples  when  equalization  is  poor.  Also, 
when  the  equalization  in  the  digital  algorithm  is  fairly  accurate, 
the  taps  are  not  forced  to  change  as  often  as  with  the  other,  thus 
providing  greater  accuracy. 
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SECTION  IV 


4.0  FULL  EQUALIZER^ 

Equation  (13)  helps  illustrate  that  to  maximize  the  eye  opening, 
one  must  reduce  the  cross  correlation  between  the  transmitted  message 
and  the  quantization  error  in  the  equalized  signal.  Appendix  C  shows 
that  to  minimize  the  mean-squared  value  of  the  quantization  error 
mentioned  abo\c,  one  must  reduce  the  cross -corre la tion  between  the 
error  and  the  unequalized  input  signal.  Thus, 

Ac^  =  -  e  x.  ,  e .  (14) 

k  j  -k  j 

The  full  mean-squared  error  equalizer  is  illustrated  in  Figure  2. 

In  Appendix  D,  the  mean-squared  error  algorithm  is  shown  to  always 
converge  when  e  is  chosen  properly. 

4.1  '‘Hybrid"  Algorithm"6- 

The  major  drawback  of  the  MSE  equalizer  illustrated  in  Figure  2 
is  the  necessity  of  two  multiplications  and  an  accumulation  for  each 
tap  during  a  single  baud.  To  reduce  this  complexity,  the  Hybrid 
algorithm  calls  for  truncation  of  the  error  signal  to  +  e.  Thus, 
the  algorithm  for  tap  update  would  be: 


ex  SGN  (e.) 
J-k  J 


(15) 
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Figure  2  SAMPLED  DATA  MODEL  OF  SYSTEM 


Hirsch  and  WolfL  have  results  indicating  that  this  algorithm 
does  converge,  but  not  as  fast  as  the  full  MSE--logical  because 
information  is  discarded  in  taking  the  sign  of  the  quantization 
error. 

4.2  Modified  Zero-Forcing11  Algorithm1-^ 

L6] 

This  is  the  algorithm  preferred  by  Hirsch  and  Wolf  because 
it  is  as  easy  to  implement  as  the  digitized  adaptive  zero-forcing 
algorithm  of  Section  3.2,  yet,  seems  to  always  converge  (although 
not  quite  as  fast  as  the  two  previous  MSE  algorithms).  The 
simplification  is  the  truncation  of  the  tap  signal  that  is  to  be 
correlated  with  the  truncated  error  signal: 


e  SGN  (x.  .  )  SGN  (e.) 
j-k  J 


(16) 


Thus,  an  up-down  counter  can  be  used  here  to  control  the  tap  gain 
settings . 
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SECTION  V 


5.0  COMPUTER  SIMULATION 

Simulation  of  a  realistic  data  transmission  system  having  a  mean 
squared  error  equalizer  (Section  4.0)  were  run  on  the  IBM  360/50  in 
the  Fortran  IV  language.  Figure  3  is  a  model  of  that  system 
(see  MTR-893).  Its  components  were  simulated  as  follows: 

(1)  The  binary  message,  a^  =  +  1,  is  a  random  binary  sequence 
without  any  self  correlation  (a^  can  also  be  taken  from  a 
multilevel  signal  set  containing  2n  levels  where  n  is  the 
number  of  bits  in  the  message).  This  constraint  is 
necessary  for  the  equalizer  to  adapt  for  all  frequencies. 

(2)  Wave  shaping  is  accomplished  by  using  the  time  domain 
representation  of  the  waveform,  in  this  case,  a  signal 
whose  Fourier  transform  is  a  raised  cosine  pulse--a  periodic 
signal  with  no  intersymbol  interference  at  its  sampling 
time.  Time  truncation  is  also  performed--a  practical 
necess ity. 

(3)  After  applying  the  fast  Fourier  transform,  the  signal  is 
lowpassed  (truncated  in  the  frequency  domain)  to  prevent 
aliasing  when  modulating. 

(4)  Modulation  is  accomplished  by  merely  translating  in  the 
frequency  domain. 
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Figure  3  MEAN- SQUARED  ERROR  EQUALIZER 


(5)  Because  of  limited  channel  bandwidth,  only  the  lower  side¬ 
band  is  to  be  sent.  This  is  accomplished  using  a  filter 
with  symmetric  rolloff  about  the  carrier,  in  this  case: 
raised  cosine  rolloff.  To  partially  compensate  for  the 
line,  predistortion  was  introduced. 

(6)  The  channel  model  was  limited  to  linear  distortion.  This, 
again,  was  accomplished  in  the  frequency  domain. 

(7)  Coherent  demodulation,  a  necessity  for  VSB ,  was  accomplished 
by  using  the  phase  of  f  ,  the  carrier  frequency,  and  then 
translating  down  in  frequency  after  adjusting  for  phase. 

(8)  Noise  was  added  in  the  time  domain  after  taking  the  inverse 
fast  Fourier  transform  of  the  demodulated  signal, 

.9 

(9)  The  equaliser  used  has  21  taps;  e  varied  between  2  and 
2  where  e  is  the  gain  in  the  tap  gain  update  loop. 

After  the  simulations  were  run,  it  was  decided  that  one  system 
configuration  improvement  would  consist  of  using  a  matched  filter  at 
the  receiver.  Thus,  the  waveshaper  would  send  a  signal  whose  trans¬ 
form  is  aj'rCO  and  the  receivers  filter  would  be  the  same- -a  1  lowable 
because  H(f)  is  pure  real.  One  advantage  of  having  a  matched  filter 
is  an  improved  signal  to  noise  ratio. 
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Unfortunately,  the  running  time  for  the  above  simulation  is 
excessive.  One  thousand  iterations  (.3125  sec  real  time)  ran  for 
approximately  forty-five  minutes.  Convergence  had  not  yet  been 
achieved.  Thus,  one  is  compelled  to  re-evaluate  the  trade-off 
between  computer  simulation  and  hardware  experiments.  One  obvious 
approach  is  to  fabricate  a  hardware  model  of  the  complete  modem. 
However,  this  would  be  expensive  and  time  consuming.  A  more  flexible 
and  cost  effective  approach  is  the  hybrid  combination  of  computer 
simulation  to  generate  the  equivalent  receiver  signal  waveform  as 
it  would  exist  just  prior  to  equalization  and  then  convert  this 
signal  (by  appropriate  D/A  conversion)  to  an  analogue  signal  input 
to  a  breadboard  model  of  the  proposed  equalizer.  The  necessary 
equipment  to  implement  such  a  process  is  already  available  or  being 
procured . 
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SECTION  VI 


6.0  OPTIMIZATION  OF  SAMPLING  TIME 

Transversal  equalizers,  by  their  very  nature,  improve  the 
probability  of  error-free  detection  in  a  highly  localized  region 
about  the  sampling  instant.  Phase  jitter  will  severely  degrade  the 
performance  provided  through  equalization  by  causing  the  receiver 
to,  in  effect,  sample  at  times  other  than  the  proper  sampling  time. 
Appendix  E  shows  how  bad  the  degradation  can  be.  Consequently,  it 
is  important  to  know  the  amount  of  phase  jitter  at  any  given  instant 
in  order  to  sample  at  the  correct  time.  Accurately  tracking  the 
phase  jitter,  however,  is  a  major  problem. 

In  order  to  achieve  higher  data  rates,  it  is  necessary  to  use 
as  much  bandwidth  as  possible.  For  PAM-VSB  (supressed  carrier) 
modulation  schemes,  a  carrier  somewhere  in  the  range  2-3  KHz  will 
almost  always  be  in  the  midst  of  data.  Consequently,  accurate 
tracking  of  the  carrier  phase  is  not  possible  here  because  of 
interference  caused  by  the  signal.  (One  must  track  with  a  bandwidth 
of  approximately  600  Hz.)  Some  modulation  schemes  shape  the 
transmitted  signal  by  inserting  a  notch  at  the  carrier  to  allow 
the  addition  of  a  pure  carrier  tone.  Because  of  the  necessity  of 
wide  tracking  bandwidth,  this,  too,  has  proven  to  be  unsuccessful. 
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SECTION  VII 


7.0  CONCLUSION 

The  bulk  of  this  work  has  been  devoted  to  the  presentation  of 
the  current  state-of-the-art  of  equalization.  For  a  data  quality 
telephone  channel,  the  algorithm  that  is  superior  in  terms  of  both 
cost  and  effectiveness  has  not  yet  been  determined.  In  addition, 
in  order  to  achieve  sufficiently  high  data  rates,  it  will  be 
necessary  to  compensate  for  phase  jitter.  Such  compensation  may 
be  integrated  into  an  equalization  algorithm,  both  of  which  require 
further  study. 
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APPENDIX  A 


Convergence  of  the  Iterative  Preset  Algorithm 
The  algorithm  for  change  of  tap  gain,  c^ ,  is  given  by  equation  (12) 


Ac.  =  -  e  SGN  q . 
J  J 


(A- 1) 


or 


A  £  =  -  e  SGN  £ 
From  equations  (2)  and  (11a): 


(A-2) 


£  =  X  £  -  u 


,  (n+1)  (n+1) 

Thus ,  £  =  X  £  -  u 


(n+1)  _  v  ,  n  * 

£  -X(c+Ac)-u 


(A-3) 
(A-4) 
(A- 5) 


From  equation  (A-2): 


£(n+1)  =  x  c(n)  -  e  X  SGN  -  u 


(A-6) 


From  equation  (A-3): 


^(n+1)  =  _  e  x  SGN  cL(n) 


(A-7) 
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or 


(n+l)  _  (n) 

qk  "  ’k 


n 

e  Y,  (SGN  ^.j 

j=-n 


(A-8) 


(n+l)  =  (n) 

^k  ^k 


e  xQ  SGN 


n 

=  £  (SGN  xk_. 

j=-n 

j^k  (A-9) 


The  normalized  distortion  of  x(t)  before  equalization  is  defined  as 


D 


i 

j=-°° 

j^o 


(A- 10) 


where  x  is  assumed  to  be  positive.  Using  Schwarz's  Inequality 

o 

on  equation  (A-9): 


hkn+1)Mqkn>  -  e  xQ  SGN  q£n)|  +  e|  £  (SGN  q™ )  |  (A-11) 

j=-n 

Note  the  following: 


U'n)  -  e  k0  SGN  q'n)|-||q£n)|  -  c  *0|  (A-12) 
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(A- 13) 


|  t  <s™  ^n>)Vjls  t  I  (SGN  SlVjl*  E  1^1 

j=-n  j=-n  j  =-n  j=-oo 

j^k  ji^k  j£k  j#) 


Thus,  using  equations  (A-10)  ,  (A-ll)  ,  (A-12)  and  (A-13): 

lqkn+1)Ullqkn)|  -  e  xQJ  +  e  xQ  d  (A- 14) 

If  the  eye  is  assumed  to  be  initially  open,  D  <  1;  hence 

lqk°+1)  l<l  lqkn)  |-  c  X0I+  6  xo  (A"15) 

If  |q£n^|>exQ,  equation  (A-15)  becomes 

'qkn+1)|<lqkn>l  (A’16) 

If  |q^n^|<exQ  ,  equation  (A-15)  becomes 

|q]Jn+1)|<2Cx0-|qk|  (A- 17) 

<2exQ  (A- 18) 
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Thus,  when  the  eye  is  initially  open,  e^ch  will  converge  tv) 
•within  of  zeio. 
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APPENDIX  B 


Estimation  of  q. 

2l 

When  transmitting  data,  the  output  is  due  to  past  and  future 
input  pulses  as  well  as  noise: 


=  T  a,h,  .  +  n,  -  a  q,  +  a,  +  n 
^  j  k-j  k  L*  j  Hk- j  k  l 


(B- 1) 


1  =-00 


where  the  a,!s  are  the  input  amplitudes  and  the  n^'s  are  noise 

samples.  One  assumes  that  (1)  the  n  *s  are  independent,  zero- 

K 

2 

mean  Gaussian  r.v.  's  of  variance  a  and  (2)  the  's  are 
uncorre la ted . 

The  joint  probability  density  of  K  samples  of  y  ,  given  the 
set  of  q  *s  is  given  by: 


K 

r>(z.|a)  =  n 

k=l 


exp[- t?  'VV.SjjV^2 


2a 


j=-‘ 


\,2tt 


(B-2) 
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Thu  likelihood  function,  L(y_|£)  =  In  p  ,  is  given  by: 


K 


L(z|s)  ■  E  -  ~T  -  ak  '  E  aiqk-1)'  "  I  ln(2na')  (B'3) 

k-1  20  j— 


Our  maximum  likelihood  estimates  of  are  given  by  setting 

^  =  0: 

uqi 

r  ■  h.  (yk  -  ak  - 1  aj  \.j)  ak-i =  ° 

k=l  i =-« 


(B-4) 


K 


K  ® 


z  ^ 


k-1 


a,  )  a.  .  -  V  V  a.  q,  .a,  . 

k'  k-i  L  Lj  j  ^k-j  k-i 

k-1  j=-“ 


=  0 


(B-5) 


Assuming  that  the  a^  's  are  uncorrelated  and  that  the  length  of 
averaging  period,  K,  is  sufficiently  long,  one  can  assume  that 


K 


y  a,  .a.  .  =  KS  6 
L>  k-j  k-i 

k=l 


ij 


(B-6) 


where  S  is  the  average  signal  power.  Using  equation  (B-6)  and 

A 

defining  -  a^,  one  gets 


t 


A 

k  “k-i  4-  q j  "ij 


e.  a 


KS6  .  .  =0 


(B-7) 


k=l 


28 


Solving: 


k=l 


e  a 
k 


k-i 


(B-8) 
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APPENDIX  C 


Minimization  of  Mean-Squared  Error 

Figure  4  shows  a  sampled  data  model  of  our  system.  Our  goal 
is  to  minimize: 


2  /  N2 

eJ  -  <*J  -  V 


(c-i) 


where  overbar  indicates  mean  value.  From  Figure  2: 


=  C  E  cv^  Vj-V-W  +  nj-v]  -  aJ^ 

v=-n  w=-cc 


(C-2) 


—  n  n  1 

e2=sr  c  f  V  g  a  +n  V  c  {  K  g  a.  +n.  }-a  ] 

ej  L  O  CV1  U  j-v-w  j-vJ  jJL  ul  u  z  j-u-z  j-uJ  jJ 

v=-n  w=-»  u=-n  z=-CD 

(C-3) 

“7  n  n  00  00 

-j  =  E  E  cvCuC$nn(v_u)  +  E  Z  8w8z$aa<V+W‘U‘z)] 


v=-n  u=-n 


w=-0D 


n  03 


-  Z  c„  E  sw$aa(v+w)  -  c.t  E  Maa<u+z>+$„7°> 


v=-n  w=-® 


u  z  aa  aa 

u--n  z  =-® 


(C-4) 
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Figure  4.  SYSTEM  MODEL  FOR  COMPUTER  SIMULATION 


/UVol 


4 


where 

$  _(u)  =  n(k)n(k+u) 
nn  — 

f 

(C-5) 

i'  (u)  =  a(k)a(k+u) 
aa  ~ 

t 

(C-6) 

$  (u)  =  0 

r»  ra 

. 

(C-7) 

na 

We  want  to  minimize  the  mean  squared  error  with  respect  to  each  tap: 


=  £  Cu[*nn(k-U)  +  £  £  SwSz$aa(k-K,-u-z)]  -£  Sw$aa(^) 


u=-n 


w=-°°  z=-°° 


w=-co 


+  V  c  [$  (v-k)  +  T*  y]  g  g  $  (v+w-k-z)]  -  5"*  g  §  (k+z) 
vL  nn  7  <-■  u  &w&z  aav  7J  ^  &z  aaN  7 

(C-8) 


v=-n 


w=-°°z  =-°° 


z=-°° 


=  2  (  £  cv[$nn(v"k)+  2  £  SwSz$aa(v4v-2-k)]  gz$aa(z+k)} 

v=-n  w=-°°  z=-co  z  =-°° 

(C-9) 

Changing  indices: 


■  2t  £  [«„n<v-k)+£iaa(2)  I  -  Z  5aa(z>8z-k! 

v=-n  z=-°°  v=-oo  z=-°° 


(C-10) 
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Now,  compare  equation  (C- 10)  with  that  derived  from  the  following: 


x 


J-k 


e  , 

j 


co  n  00 

=  [  y  g .  .  a  +n.  .  ][  Y  c  (  Tg  a,  +n.  j-a  ] 
L  Lj  &j-k-u  u  j-kJL  ^  vl  ^  w  j-v-w  j-v  j 

u=_o3  v=-n  w— 


(C-ll) 


00 


x.  .  e  . 
J-k  J 


-  E 


oo 


c  (v-k)+  g  E 

v  nn  *-*  *-* 

v--n  u=-°°  w=-~° 


5  8-  i  $  (v+w-j+u)] 

w j -k-u  aa 


-  Y  g.  1  4  (j-u) 

4-‘  j -k-u  aa  J 
u=-°° 


(C- 12) 


Changing  ind  ices : 


>kej 


=  E  ^nn(v'k)+^  $-(z)  E 


aa 


^w^v+w-z -k  jCv 


-  E 


4aa(z)8z-k 


V  =  -ll 


z=-« 


w=-< 


Sck 


(C- 13) 
(C- 14) 
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APPENDIX  D 


Convergence  of  the  MSE  Algorithm 


The  MSE  algorithm  calls  for  the  following  (equation  C-13): 


v  -n 


[y  (v-k)+  $  (2)  V1  g  g  .  ,  ]c  -  y  §  (z)g  =0 

nn  aa '  i~>  °w  v+w-z-k  v  6  m  aa  z-k 

z  =- 00  w=-°°  z =- 00 

(D- 1) 

here  -  n  <  k  <  n. 


This  can  in  turn  be  written  using  matrix  notation: 


(o  +  G)  c  =  d 
n  —  —  — 


(D-2) 


or 


B  c  =  d 


(D-3) 


Our  equalization  algorithm,  equation  (14),  becomes: 


n+1  n  n 

c  =  c  -  e  (B  c  -d) 


(D-4) 


n+1  /-,  n  , 

c  =  (I  -  e  B)  c  -  ed 


(D-5) 
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Convergence  occurs  if  I  -  e  B  has 
unit  circle.  All  eigenvalues  are 
hence,  the  eigenvalues  of  B  mus 

0  <  ^  <  2/e 


all  eigenvalues  within  the 
real  because  B  is  symmetric, 
t  satisfy  the  following: 


(D-6) 


Energy  considerations 
while  proper  choise  of 
smaller  than  the  upper 


provide 
e  will 
bound . 


for  satisfaction  of  the  lower  bound 
gurantee  that  the  eigenvalues  are 
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APPENDIX  E 


A 


* 


ilourd  on  Probability  of  Error  for  a  Given  Amount  of 

Distortion  In  an  8-Level  System 
For  an  8- level  system,  the  various  levels  and  thresholds  are 
ind  ica  ted  be  low: 


I.  TLTLTLTLTLTLTL 

•t — i — i — i — i — i — i — i — i — i — i — i — i — i — t — > 

-6-5-4-3-2-1123456 

777777^777777 


Additive  noise  has  the  Gaussian  density  function: 


Vx) 


(E  — 13 


Thus,  in  the  absence  of  distortion,  the  probability  of  error  is 
given  by: 

1 

'  7 

pc  =  \  C  r  PN  (*)  dx  +  f  PN  (x)  dx]  (E-2) 

1 

7 

wh ^re  the  factor  of  \  is  due  to  the  absence  of  thresholds  above 

o 

the  highest  level  and  below  the  lowest  level  and  also  to  the 
assumption  that  each  level  is  equally  likely.  When  distortion  is 
present,  it  serves  to  displace  the  density  to,  in  effect,  give  it 
a  non-zero  mean: 
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p  = 


i  r 


(X-D) 

3 


(X-D)' 


V-TT  0. 


dx  + 


-v/^TT 


dx  J  ,  (E-3) 


'N 


where  D  io  Lhe  amount  of  distortion  normalized  to  the  two  extreme 

X-D 

levels.  Change  variables  (y  =  - )  : 

°N 


P  =  a 


l  r  -L  r 


-1-7D 

7°N  -y2/ 2 


dy  +  - 


e'y  72  dy 1 


1-7D 

7  c„ 


(E-4) 


P.  = 


8v  2 "  1+7D 


7  c 


'  2  dy  +  J*  e"y  /2  dy] 

1-7D 


N 


7  a 


N 


(E-5) 


7  ,  ,  1+7D.  r  ,1-7Dn 

pe  =  8  !crfc*  C— >  ^  erfc*  C— )  1 

CN  N 


(E-6) 


r  141 

here  e r ic,  (:)  is  defined  on  page  37  of  Van  Trees.  J 
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Figure  5  is  a  plot  of  p  vs.  D  for  various  S/N  ratios  that 

e 

demonstrates  how  harmful  distortion  can  be. 

For  a  A- level  system,  the  probability  of  error,  given  by  an 
equality  similar  to  equation  (E-6)  is 


3  r  .  ,  l+3Ds  .  . 

P„  -  T  Cerfc*  (-TT-)  +  erfc 


3  a 


N 


*v  3a. 


N 


(E-7) 


here  D  is  the  distortion  normalized  to  the  extreme  levels  (+1)  - 
i  f  D  =  1 ,  the  binary  eye  is  closed.  Figure  6  gives  the  p^  vs.  D 
plots  for  4-level  transmission. 
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Pe,  PROBABILITY  OF  WORD  ERROR 


D,  NORMALIZED  DISTORTION 


Figure  5.  Pe  vs.  D  FOR  AN  8— LEVEL  SYSTEM 


S/n  (db) 


Figure  6.  Pe  vs.  D  FOR  A  4-LEVEL  SYSTEM 
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